Increasing the Capacity of a Hopfield Network without Sacrificing Functionality

Hoppeld networks are commonly trained by one of two algorithms. The simplest of these is the Hebb rule, which has a low absolute capacity of n=(2 ln n), where n is the total number of neurons. This capacity can be increased to n by using the pseudo-inverse rule. However, capacity is not the only consideration. It is important for rules to be local (the weight of a synapse depends ony on information available to the two neurons it connects), incremental (learning a new pattern can be done knowing only the old weight matrix and not the actual patterns stored) and immediate (the learning process is not a limit process). The Hebbian rule is all of these, but the pseudo-inverse is never incremental, and local only if not immediate. The question addressed by this paper is, `Can the capacity of the Hebbian rule be increased without losing locality, incrementality or immediacy?' Here a new algorithm is proposed. This algorithm is local, immediate and incremental. In addition it has an absolute capacity signiicantly higher than that of the Hebbian method: n= p 2 ln n. In this paper the new learning rule is introduced, and a heuristic calculation of the absolute capacity of the learning algorithm is given. Simulations show that this calculation does indeed provide a good measure of the capacity for nite network sizes. Comparisons are made between the Hebb rule and this new learning rule.